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Chapter 13 Preliminary studies and pilot testing 


1. Introduction to preliminary studies and pilot testing 


The time between the idea for an intervention trial and first entering participants into the trial is usually long, 
generally at least a year and often several years. Even when funding for a trial has been obtained, which, in itself, may 
take a year or more, there is often much work to do before the first participant can be enrolled into the trial. This 
chapter outlines the kinds of investigations and studies that may be carried out before starting the main trial to try to 
maximize the possibility that the trial will be conducted successfully. We divide these into two kinds of study. First are 
preliminary studies to develop different aspects of the trial procedures or to collect data to facilitate the planning and 
conduct of the trial. Second are pilot studies which are tests of the full trial procedures on a small sample of potential 
participants to make sure, in so far as is possible, that any problems with the conduct of the trial will be identified, so 
that procedures can be changed before the full trial starts. 


Though often very useful, no specific type of preliminary study is invariably essential, whereas a pilot study should 
always be planned, though such studies can range from a relatively brief testing of the intervention and its evaluation 
that lasts a week or less through to an extensive period of testing and refinement of the intervention and evaluation 
methods that spans several months, or even a year or more. 


2. Preliminary studies 


2.1. Purposes 


Preliminary studies are often conducted to refine the intervention and evaluate its acceptability, feasibility, cost, and 
uptake. For example, prior to a large field trial of a multi-component intervention that aimed to improve adolescent 
sexual and reproductive health in Tanzania, a preliminary study was carried out to test and refine the intervention. The 
main cluster randomized trial was planned to involve about 10 000 adolescents in over 120 schools, with an initial 
follow-up period of 3 years. A preliminary study was conducted to develop and refine the intervention methods that 
would be used to train and support teachers and class peer educators who would deliver the in-school sexual and 
reproductive health education intervention to be used in the trial (Obasi et al., 2006). 


Preliminary studies may be needed to provide local up-to-date data, in order to calculate or confirm the sample size 
required for the main trial. For example, before embarking on a field trial of a malaria vaccine that will be evaluated 
for its effect in reducing the incidence of clinical cases of malaria, a preliminary study may be required to obtain 
estimates of the incidence of cases of malaria in the study population, probably spanning a complete year, in order to 
allow for seasonal variation in transmission. The outcome from such preliminary studies provides the data necessary 
for designing the size of the main trial. It is commonly found in trial design that investigators are over-optimistic 
about the likely frequency of outcome events in their trial population. Consequently, after a preliminary (baseline) 
study, the size of the main trial needs to be increased. Sometimes, the reverse happens, but not so commonly! In so far 
as is possible, the baseline study should be conducted under similar conditions to those that will hold in the main trial. 
Thus, for example, if insecticide-impregnated bed-nets are to be distributed to all children participating in a trial of a 
malaria vaccine, as may be required for ethical reasons, this should be done for the baseline studies to avoid over- 
estimating the likely incidence of malaria in the trial population (Leach et al., 2011). 


In some cases, preliminary investigations may even show that the proposed study population will not be suitable. A 
trial of a vaginal microbicide gel to prevent HIV transmission among women in Ghana was based upon an assumption 
of an annual transmission rate of HIV in the trial population of 5% a year. Baseline studies were not conducted to 
verify this assumption, and, once the trial had started, it was discovered that the actual transmission rate was only 
about 1% a year. Thus, an expensive trial had to be abandoned, because of a lack of statistical power (Peterson et al., 
2007). Had it been known, before the trial started, that it should have been five times as large, it perhaps would never 
have been started. 


Preliminary studies may also be needed to estimate how long it will take to enrol the target number of trial 
participants, the proportion of participants who are likely to be lost to follow-up, the best interval to have between 
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follow-up visits, and the overall duration of the trial. 


Other preliminary studies are helpful to refine the design of specific methods for use in the process and/or impact the 
evaluation within the main trial, and to evaluate their acceptability, feasibility, and cost. For example, will taking 
blood specimens, skin snips, or self-administered vaginal swabs be feasible and acceptable? Can the cold chain be 
maintained for vaccines or specimens that need to be kept cold, and for how long, since this will govern how 
frequently they need to be taken to or from the field research team? How many staff will be required, and how much 
will it cost, to carry out and collect data and specimens from 60 participants a day, for example? 


It will also be necessary to explore the likely community acceptance of the trial (see Chapter 9), staff training needs, 
and other logistic requirements related to field and laboratory activities, data management, and study clinics. Some of 
these data may have already been collected in studies previously conducted by the trial team or by others, but, in other 
circumstances, special preliminary studies are required. 


Many preliminary studies can be small and quick such as a qualitative study to ask potential trial participants to 
review a draft information sheet for clarity and acceptability. On the other hand, others may take over a year such as a 
study to check the incidence of a seasonal disease that must cover at least one 12-month period. 


An example of a relatively large preliminary study conducted prior to a trial was the feasibility study for a multicentre 
trial of the impact of a vaginal microbicide on HIV incidence among women at high risk of acquiring sexually 
transmitted infections (STIs) that was conducted in four East and Southern African countries (McCormack et al., 
2010). The main trial was planned to be conducted over several years at a likely cost of tens of millions of pounds, so 
it was crucial to ensure, prior to starting the main trial, that the sample size was right and that the methods planned for 
all aspects of the trial were both feasible and acceptable. Within the Tanzanian site for the trial, for example, a 
preliminary study was designed (Vallely et al., 2007). This lasted more than a year, to: 


identify the population groups to invite to participate in the trial 


work out how best to deliver the intervention and related clinical services 


+ 


evaluate the likely acceptability of the microbicide gel 


+ 


test and refine the study methods and instruments 


+ 


estimate the incidence of the primary (HIV) and secondary (STIs, reported use of the microbicide gel) outcomes 
for the main trial, and 


@ estimate the costs of each of the activities needed for the trial. 


2.2. Design of preliminary studies 


The design and methods used for preliminary studies should be tailored to address the specific issues and questions to 
be answered. Often, both qualitative and quantitative methods will be required, drawing upon social and behavioural 
sciences, and economic, epidemiological, laboratory, statistical, and community development approaches. Usually, a 
preliminary study will be relatively short term and inexpensive, in comparison to the main trial. Ideally, the main trial 
should be started soon after the preliminary study to avoid the situation changing between the two. This frequently 
raises the question of whether preliminary studies should be built into the funding proposal for the main trial, or 
whether they should be the subject of one or more separate preliminary funding proposals. If the latter approach is 
adopted, there may be a delay between the preliminary investigations and funding being secured for the main trial. A 
reasonable approach might be to present the design of the main trial to the funding agency, but acknowledging that 
preliminary studies will be necessary to confirm some of the assumptions in the proposal such as disease incidence 
rates. The funding for the main trial might then be made conditional on the results of the preliminary investigations. If 
the preliminary studies indicate that additional funding will be required for the main trial, for example, because the 
sample size has been underestimated, then the agency may wish to reconsider the proposal. The best strategy will 
often depend on the work that has been done in the past and the degree to which the results of the preliminary studies 
might affect the size, duration, or cost of the trial. Further details on some of the social and behaviour science methods 
that can be used within preliminary studies can be found in Chapter 15. 


It is usually best to conduct the preliminary studies in the same general population, but in different individuals (or 
clusters) from those who will be involved in the main trial. 
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A preliminary study for the in-school intervention component of the Tanzanian adolescent sexual and reproductive 
health trial mentioned in Section 2.1 was conducted over a period of about 6 months in five schools that would not be 
included in the subsequent trial but that were conveniently located close to the offices of the research institution 
coordinating the trial. Teachers and class peer educators were selected and trained to deliver the in-school sessions 
and were then observed actually teaching the sessions to evaluate the session quality and how long it took to teach 
each session. The study identified misunderstandings and that there were some topics that the teachers obviously felt 
uncomfortable teaching, for example. Researchers also interviewed the teachers, peer educators, school headteachers, 
some of the students, and their parents to get their impressions of each session and the course as a whole and their 
suggestions for improvements. In the course of this preliminary study, many lessons were also learned about the 
resources that would be needed, the best ways to select the teachers and peer educators, and how to gain the trust of 
the local education department, school authorities, local religious leaders, students, and their parents. 


The feasibility study in the Tanzania site of the microbicide trial mentioned in Section 2.1 involved conducting a rapid 
assessment and mapping of bars, guesthouses, restaurants, shops, sellers of local brew, and wayside food sellers, and 
enumeration of the number of women working in them to identify the potential numbers that could be invited to join 
the subsequent trial. A group of these women were invited to join a preliminary longitudinal cohort study which 
would receive all the proposed trial procedures, except being given either the microbicide or placebo gel. The 
procedures included setting up study clinics that the women were asked to attend on a quarterly basis and the regular 
monitoring of the outcomes that were proposed for the trial, including tests for HIV and other STIs, pregnancy, and 
reported sexual behaviours. The opportunity was taken to conduct comparisons of alternative ways of collecting data 
on self-reported sexual behaviours (including face-to-face interviews and use of pictorial diaries kept by the women) 
and of testing various alternative methods for interacting and exchanging information with women participants, their 
representatives, the owners and managers of the institutions in which they worked, community leaders, and relevant 
local officials. Discussions and negotiations were held with health facilities where women were referred for clinical 
care beyond the scope of the trial team themselves. The feasibility study also allowed detailed preparations and 
negotiations with national and international regulatory authorities. 


Pre-testing of procedures for data and specimen collection and analysis should always be part of the preliminary 
studies for a trial. For example, any information sheet or questionnaire should be translated and back-translated if it is 
to be administered in a different language from the original in which it was designed. If it is going to be administered 
in several different languages, this can take a considerable amount of organization and time. The document should be 
pre-tested by administering it to a small number of volunteers. This will usually reveal problems with the order or 
clarity of information or questions, or with the coding of answers. Clearly, enough time must be left to act on the 
lessons learned during the pre-testing, and it may be necessary to pre-test several sequential versions of an 
information sheet or data collection form, before it is considered ready for pilot testing. More details on questionnaire 
design are given in Chapter 14. 


3. Pilot testing 


3.1. Purpose 


Every field trial should be preceded by a pilot study (also known as a pilot test) prior to launching the main trial. This 
should test, on a small scale, all the study procedures, including the selection of eligible potential participants, their 
enrolment, recording the required data, specimen collection (if applicable), supervision systems, quality control, and 
data processing. If the trial involves multiple data collection rounds, where either staff or procedures change between 
rounds, it is a good idea to pilot test the procedures before each round. 


3.2. Design of the pilot test 


The design of the pilot study should be as similar as possible to the design of the procedures in the main trial, and the 
population selected to take part should be representative of the trial population (though not part of it). In a drug or 
vaccine trial, the actual interventional and comparison products (for example, drug or vaccine or placebo) might be 
administered, and procedures tested for monitoring immediate outcomes and responding to any potential AEs. 
However, sometimes, only the standard comparison product or placebo is used in the pilot study, as those included in 
the pilot study might not be included in the long-term safety monitoring that would be present in the main trial. For 
example, only the placebo gel was used in the pilot test for the microbicide trial described in Sections 2.1 and 2.2. For 
other types of intervention, such as the combination of in-school sexual and reproductive health education, training of 
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health workers and youth condom promoters, and community-wide supportive activities that were evaluated within 
the trial that was also mentioned in Sections 2.1 and 2.2, the interventions were pilot-tested in separate communities. 


Usually, it is best to conduct the pilot study in individuals or a cluster that will not be included in the main trial, in 
order to avoid having to go back to the same individuals to collect similar data in the main trial. In a multi-round trial, 
the same specific individuals or clusters might participate in the pilot test that precedes each data collection round. 
This has logistic advantages. The field teams will get to know the community in which the pilot tests are conducted, 
facilitating logistics such as where to conduct the survey, where to stay overnight, and who the best local people are to 
ask to help introduce the study to householders or to help find people who do not come forward for the trial. It also 
has the technical advantage that the individuals and communities involved in subsequent rounds of the pilot test will 
have had similar prior exposure to the procedures to those in the main trial population. 


The pilot study can often be linked to staff training. For example, in a multi-round field trial of vitamin A 
supplementation in children, staff received a specific training course that covered all the field data collection methods 
that would be used in the subsequent trial round. This course lasted a total of 2 weeks and included both classroom 
and practical training. During the first week, the practical training included ‘mock interviewing’ their colleagues and 
role plays, in which one interviewer asked questions of the trainer, while all the field interviewers entered the answers 
into the questionnaire. The pilot test was carried out early in the second week, so that any necessary changes could be 
made to the procedures, or even to the data collection forms, in time for the interviewers and their supervisors to be 
brought up to speed on the modifications before the end of the 2-week training period. 


Every step in the field trial processes should be tested in the pilot study. Importantly, the pilot test of data and 
specimen collection procedures must allow enough time for the pilot data to be entered on to computers, ‘cleaned’, 
and analysed, so that these systems can also be checked for functionality. Similarly, whenever possible, any specimens 
collected during the pilot test should be processed, so that, at a minimum, it is possible to check that the specimens 
have been collected and transported correctly and are in good condition. In addition, enough time must be allowed 
between the completion of the pilot test and all its checks, for revisions to be made to the instruments and procedures 
if they are needed. All too often, inexperienced trial managers do not allow enough time for this and hope that no 
changes will be needed or are then under pressure to ignore indications from the pilot test that improvements would 
be desirable. 


Sometimes, investigators are tempted to use the results from a small, time-limited pilot test to predict whether the 
sample size that was calculated for the main trial will be sufficient. While a small pilot test can give rise to worries 
about recruitment rates and suggest ways of increasing these, pilot studies will usually not have been designed with 
sufficient numbers or duration to give a precise enough estimate of trial outcomes to make it sensible to attempt to use 
it to test sample size calculations. Given very wide CIs around the outcome estimates that are likely in a small pilot 
test, such projections may be very misleading. If there is a need for checks on the assumptions used in the trial sample 
size calculation, these should be tested within a preliminary study, as described in Section 2.2. 
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